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[Name of the Document] Specification 

[Name of the Invention] Imaging Apparatus f Imaging Method, 
Signal-recording Apparatus, and Signal-recording Method 
[Scope of Claim for a Patent] 

[Claim 1] An imaging apparatus characterized in that 
the imaging apparatus comprises: 

imaging means for reading a signal out of an image 
pickup device so as to generate an image signal based on 
captured image; 

signal-processing means for generating image data 
based on said image signal; 

control means for controlling operation of said 
imaging means and said signal-processing means to change a 
frame rate of said image data to a desired set frame rate, 
and generating associated information for indicating said 
set frame rate; and 

transmitting means for combining said associated 
information with said image data to transmit them. 

[Claim 2] The imaging apparatus according to Claim 1, 
characterized in that said control means changes the frame 
rate of said image data by means of altering timing of 
reading the signal from said image pickup device. 

[Claim 3] The imaging apparatus according to Claim 1, 
characterized in that said control means changes the frame 
rate of said image data by means of controlling said 
signal-processing means to perform frame-skipping. 

[Claim 4] The imaging apparatus according to Claim 1, 
characterized in that said control means changes the frame 
rate of said image data by means of altering timing of 
reading the signal from said image pickup device and 
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controlling said signal-processing means to perform 
addition on a frame basis. 

[Claim 5] The imaging apparatus according to Claim 1, 
characterized in that said control means adds a sub-frame 
number to each of the frames of said set frame rate 
included within one frame period of reference frame rate 
so as to include said sub-frame number in said associated 
information . 

[Claim 6] The imaging apparatus according to Claim 1, 
characterized in that said signal-processing means samples 
analog audio signal to generate audio data; 

said control means controls a sampling frequency of 
said analog audio signal in said signal-processing means 
based on said set frame rate; and 

said transmitting means combines said associated 
information with said image data and said audio data to 
transmit them. 

[Claim 7] An imaging method characterized in that the 
method comprises the steps of: 

reading a signal out of an image pickup device to 
generate an image signal based on captured image; 

changing a frame rate of said image data to a desired 
set frame rate when generating image data based on said 
image signal, and generating frame rate information for 
indicating said set frame rate; and 

combining associated information containing said 
frame rate information with said image data to transmit 
them . 

[Claim 8] The imaging method according to Claim 7, 
characterized in that the frame rate of said image data is 
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changed by means of altering timing of reading the signal 
from said image pickup device. 

[Claim 9] The imaging method according to Claim 7, 
characterized in that the frame rate of said image data is 
changed by means of performing frame-skipping. 

[Claim 10] The imaging method according to Claim 7, 
characterized in that the frame rate of said image data is 
changed by means of altering timing of reading the signal 
from said image pickup device and performing addition on a 
frame basis. 

[Claim 11] The imaging method according to Claim 7 , 
characterized in that a sub-frame number is added to each 
of the frames of said set frame rate included within one 
frame period of reference frame rate so as to include said 
sub-frame number in said associated information. 

[Claim 12] The imaging method according to Claim 7 , 
characterized in that analog audio signal is sampled so as 
to generate audio data, a sampling frequency of said 
analog audio signal is controlled based on said set frame 
rate, and said associated information is combined with 
said image data and said audio data so as to transmit them, 

[Claim 13] Signal-recording apparatus characterized 
in that the apparatus comprises: 

signal-recording-and processing means for generating 
a record signal from record data which combines main data 
indicating audio and/or video with associated information 
containing frame rate information of the main data; 

recording means for recording on record medium the 
record signal generated in the signal-recording-and- 
processing means; and 
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control means for controlling the signal-recording- 
and processing means and the recording means. 

[Claim 14] Signal-recording method characterized in 
that the method comprises the steps of: 

generating a record signal from record data which 
combines main data indicating audio and/or video with 
associated information containing frame rate information 
of the main data; and 

recording on record medium the record signal. 
[Dei:a±led Description of the Invention] 
[0001] 

[Technical Field Pertinent to the Invention] 

The invention relates to an imaging method, an imaging 
apparatus, signal-recording apparatus, and signal-recording method. 
Particularly, it relates to an imaging method and the like of 
reading a signal out of an image pickup device to generate an image 
signal based on captured image, changing a frame rate of said 
image data to a desired set frame rate when generating 
image data based on said image signal, and generating 
frame rate information for indicating said set frame rate, 
and combining associated information containing said frame 
rate information with said image data to transmit them, or 
recording the image data with which the associated 
information is combined on record medium. 
[0002] 
[Prior Art] 

In generation of contents on audio and/or video for 
broadcasting, it has been often done to generate the 
contents altering motion speed partially, in order to 
obtain enhanced effects the creator wishes to create. 
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[0003] 

In such the generation of contents altering the 
motion speed, for example, a higher frame rate relevant to 
a reference frame rate is illustratively set to generate 
the contents, and then the contents are reproduced at the 
reference frame rate so that slow motion contents can be 
produced. Alternatively, a lower frame rate relevant to a 
reference frame rate is set to generate the contents, and 
then the contents are reproduced at the reference frame 
rate so that quick motion contents can be produced. 
Further, by adapting the frame rate to be set and the 
frame rate at the reproduction, it is possible to alter 
the motion speed freely. 
[0004] 

Thus, the creator creates the contents for 
broadcasting using not only the contents generated at the 
reference frame rate but also the contents of altered 
motion speed, in order to obtain the enhanced effects the 
creator wishes to create when the contents are reproduced 
at the reference frame rate. 
[0005] 

The video camera allowing time axis to be compressed 
or extended in order to allow the contents obtained by 
thus altering the frame rate to be generated has been 
proposed in, for example, patent document 1. 
[0006] 

[Patent Document:] 

Japanese Patent Publication No. Hll-177903 
[0007] 

[Problems to be solved by the Invention] 
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When, however, the contents to be used for 
broadcasting are generated using not only the contents 
generated at the reference frame rate but also the 
contents of altered motion speed as material, as described 
above, it is not easily determined at editing which of 
frame rates each of the contents has been generated if 
merely the material image is displayed. Further, when it 
performs the speed change on the contents of altered 
motion speed, it is impossible to determine whether or not 
the speed change can be performed without any 
deterioration in image quality. For example, if a frame 
rate is set (hereinafter, referred to as, "set frame 
rate 7 ') to 10 times the reference frame rate, the 
reproduced contents are displayed one-tenth in a motion if 
the contents generated at the set frame rate are 
reproduced at the reference frame rate. Here, it is 
conceivable that since the set frame rate is 10 times the 
reference frame rate when wanting to display the contents 
one-fifth in a motion, the speed change can be easily 
performed without any deterioration in image quality if 
performing a frame-skipping for each frame. It, however, 
is impossible to determine whether or not the speed change 
can be performed without any deterioration in image 
quality, easily based on merely the displayed image. 
[0008] 

Accordingly, the present invention presents an 
imaging method, an imaging apparatus, a signal-recording 
apparatus, and a signal-recording method wherein determining 
which of frame rates the. imaging has been performed is 
easily allowed. 
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[0009] 

[Means for Solving Problem] 

An imaging apparatus according to the present 
invention comprises imaging means for reading a signal out 
of an image pickup device so as to generate an image 
signal based on captured image, signal-processing means 
for generating image data based on said image signal, 
control means for controlling operation of said imaging 
means and said signal-processing means to change a frame 
rate of said image data to a desired set frame rate, and 
generating associated information for indicating said set 
frame rate, and transmitting means for combining said 
associated information with said image data to transmit 
them. 
[0010] 

Further, an imaging method according to the present 
invention comprises the steps of reading a signal out of 
an image pickup device to generate an image signal based 
on captured image, changing a frame rate of said image 
data to a desired set frame rate when generating image 
data based on said image signal, and generating frame rate 
information for indicating said set frame rate, and 
combining associated information containing said frame 
rate information with said image data to transmit them. 
[0011] 

Further, a signal-recording apparatus comprises 
signal-recording-and processing means for generating a 
record signal from record data which combines main data 
indicating audio and/or video with associated information 
containing frame rate information of the main data, 
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recording means for recording on record medium the record 
signal generated in the signal-recording-and-processing 
means, and control means for controlling the signal- 
recording-and processing means and the recording means. 
[0012] 

Additionally , signal-recording method comprises the 
steps of generating a record signal from record data which 
combines main data indicating audio and/or video with 
associated information containing frame rate information 
of the main data, and recording on record medium the 
record signal. 
[0013] 

In the present invention, when reading signal out of 
an image pickup device so as to generate image signal 
based on captured image, and generating image data based 
on the image signal, the frame rate of the image data is 
changed by means of altering timing of reading the signal 
from the image pickup device, the frame rate of said image 
data is changed by means of performing frame- s kipping , or 
the frame rate of said image data is changed by means of 
altering timing of reading the signal from said image 
pickup device and performing addition on a frame basis. 
By changing the frame rate of the image data to a desired 
set frame rate and generating the frame rate information 
indicating the set frame rate, the image data indicating 
this image is made main data, and the associate 
information containing the frame rate information is 
combined with the main data so as to be transmitted. 
Alternatively, the associated information includes the 
sub-frame number in addition to the frame of the set frame 
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rate included within one frame period of reference frame 
rate. Further, analog audio signal is sampled so as to 
generate audio data and a sampling frequency thereof is 
controlled based on the set frame rate. The image data 
and the audio data indicating audio and video becomes the 
main data, and the associated information is combined with 
the main data so as to transmit them. Alternatively, the 
data obtained by combining the main data with the 
associated information becomes the record data, and a 
record signal is generated based on the record data and 
recorded on the record medium such as videotape and disc- 
like record medium. 
[0014] 

[Mode for Carrying Out the Invention] 

An embodiment of this invention will now be described. 
Fig. 1 shows an entire configuration of contents-providing 
system for providing the contents, for example, for audio 
and/or video. An imaging apparatus 10 generates image 
data having its changed frame rate and combines associated 
information including frame rate information indicating 
the frame rate of the image data with this image data to 
supply a signal recording apparatus 20 or an editing 
apparatus 30 with the combined ones as material data DTm . 
When the imaging apparatus 10 is provided with an audio 
input apparatus 19, it generates audio data to supply the 
signal recording apparatus 20 or the editing apparatus 30 
with the audio data and the associated information as 
material data DTm. 
[0015] 

The signal recording apparatus 20 records the 
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supplied material data DTm on recording medium. It also 
reproduces the material data DTm recorded on the recording 
medium and supplies it to the editing apparatus 30. The 
imaging apparatus 10 and the signal recording apparatus 20 
may be combined together each other as so-called 
"dockable" typed camera/recorder system, and the imaging 
apparatus 10 and the signal recording apparatus 20 may be 
separately provided with them being connected through a 
cable . 
[0016] 

The editing apparatus 30 performs editing process 
using the material data DTm supplied from the imaging 
apparatus 10 or the signal recording apparatus 20 and 
produces image data and audio data the editor wishes to 
create. It also combines associated information including 
frame rate information with the image data and the audio 
data thus produced to generate contents-data DC for 
broadcasting, distribution, and the like. 
[0017] 

Fig. 2 shows a configuration of the imaging apparatus 
10. Light incident through imaging lens system 11 is made 
incident to an imaging unit 12 and a subject image is 
formed on an image surface of an image pickup device such 
as charge coupled device (CCD) mounted within the imaging 
unit 12. The image pickup device generates imaged charge 
of the subject image though photoelectric transfer 
characteristics thereof. It also reads out the imaged 
charge thus generated on the basis of driving signal CR 
from a timing generator 142 , which will be described later, 
and generates imaged signal Sp having a frame rate that 
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corresponds to the driving signal CR to supply it to 
camera processing circuit 131 in signal processing unit 13. 
[0018] 

Based on timing signal CT supplied from the timing 
generator 142, the camera processing circuit 131 performs 
various signal processing at timings synchronized with the 
imaged signal Sp. Illustratively, noise-reduction 
processing for reducing noise components from the imaged 
signal Sp by means of correlated dual sampling or the like, 
transformation processing from the imaged signal Sp thus 
noise-reduced to digital image data, clamp processing of 
the image data, processing on shading correction and 
pilling-up of deficiency in the image pickup device, y 
correction, contour enhancement processing, knee 
correction, and the like are performed. Further, various 
signal processing are performed under the process 
conditions based on the operation control signal CS 
supplied from imaging control circuit 141 of control unit 
14. Thus, the image data DV generated according to 
various signal processing in the camera processing circuit 
131 is supplied to the transmitting unit 15. 
[0019] 

The timing generator 142 of the control unit 14 
generates driving signal CR corresponding to operation 
control signal CS from the imaging control circuit 141 and 
supplies it to the imaging unit 12, thereby allowing a 
reading-out timing of imaged charge in the imaging unit 12 
to be altered. It, then, also controls a frame rate of 
the imaged signal Sp to be equal to a set frame rate FRs 
based on an operation signal PSa from user interface unit 
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16. For example, if frame frequency, 59.94Hz or 29.97Hz 
in the case of NTSC system or frame frequency, 50Hz or 
25Hz in the case of PAL system is set to the frame 
frequency of reference frame rate FRr and an operation 
such that the set frame rate FRs is set to the one n times 
the reference frame rate FRr is carried out (k is positive 
value, not limited to an integer) , a frame rate of the 
imaged signal Sp is controlled to be equal to the one n 
times the reference frame rate FRr. 
[0020] 

The timing generator 142 also generates timing signal 
CT synchronized with the driving signal CR and supplies it 
to the camera processing circuit 131 and audio processing 
circuit 132- The timing generator 142 further generates 
frame rate information DM- FRs indicating the set frame 
rate FRs that is frame rate of image data DV and supplies 
it to the transmitting unit 15. The timing generator 142 
also generates sub-frame number BN . This sub-frame number 
BN is identification number for identifying each frame 
included in each of the frame periods of the reference 
frame rate FRr when the set frame rate FRs is set to be 
higher than the reference frame rate FRr. This sub-frame 
number BN is supplied to the transmitting unit 15 as frame 
identification information DM-BN. 
[0021] 

Fig. 3 is a flowchart showing an offer operation of 
sub-frame number in the timing generator 142. The timing 
generator 142 performs a frequency division on oscillated 
signal of a predetermined frequency, for example, to set 
the set frame rate FRs so that the frame periods of the 
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reference frame rate FRr and the set frame rate FRs may be 
synchronized with each other, and generates a frame 
reference timing indicating a generation of the driving 
signal CR and a breakpoint between the periods of frames 
of reference frame rate FRr based on the frame period of 
the set frame rate FRs. 
[0022] 

At step ST1, it is determined whether or not the 
frame reference timing is detected. If the frame 
reference timing is detected, the process goes to step ST2 . 
If no frame reference timing is detected, the process goes 
back to the step ST1. 
[0023] 

When the frame reference timing is detected at the 
step ST1, the process goes to the step ST2, as described 
above, where the sub-frame number BN is initialized, thus 
setting the sub-frame number BN to an initial value, for 
example, 0 and the process then goes to step ST3. 
[0024] 

At the step ST3, it is determined whether or not the 
frame reference timing has been detected during a period 
of time starting from detecting the frame reference timing 
and ending before one frame of period of the set frame 
rate FRs has been passed. If no frame reference timing is 
detected, the process goes to step ST4 where 1 is added to 
the sub-frame number BN and then the added sub-frame 
number BN is updated and the process then goes back to 
step ST3. Thus, when no frame reference timing has been 
detected before one frame of period of the set frame rate 
FRs has been passed, the sub-frame number BN is allocated 
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in sequence to the set frame rate FRs for each of the 

frames of periods thereof, 

[0025] 

Thereafter, when the frame reference timing is 
detected before one frame of period of the set frame rate 
FRs has been passed, the process goes back to the step ST2 
where the sub-frame number BN is initialized. 
[0026] 

Every frame of period of the reference frame rate FRr, 
the sub-frame number BN may be added to the frame image of 
the set frame rate FRs provided during a frame period of 
time . 
[0027] 

The imaging control circuit 141 of the control unit 
14 shown in Fig. 2 is connected with the user interface 
unit 16. When, in the imaging apparatus 10, switching 
operation and alteration operation of the frame rate are 
performed, the user interface unit 16 generates operation 
signal PSa corresponding to these operations and supplies 
it to the imaging control circuit 141. When supplying the 
operation signal PSa from external equipment, the user 
interface unit 16 supplies the imaging control circuit 141 
with the operation signal PSa. 
[0028] 

The imaging control circuit 141 generates operation 
control signal CS to allow the imaging apparatus 10 to be 
operated according to the operation signal PSa based on 
the operation signal PSa from the user interface unit 16 
and also supplies it to the camera processing circuit 131 
and the timing generator 142. 
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[0029] 

To the audio processing circuit 132, analog audio 
signal Sin is supplied from audio input apparatus 19. The 
audio processing circuit 132 also performs sampling 
process on the audio signal Sa based on timing signal CT 
supplied from the timing generator 142 to generate digital 
audio data DA and supply it to the transmitting unit 15. 
[0030] 

The transmitting unit 15 generates associated 
information DM including the frame rate information DM-FRs 
and the frame identification information DM-BN and 
combines it with the image data DV and the audio data DA 
to generate material data DTm and supplies it to the 
signal recording apparatus 20 or the editing apparatus 30. 
The associated data DM may include not only information on 
the set frame rate FRs and the sub-frame number BN but 
also information on imaged date and time, imaged condition, 
imaged contents , and the like . 
[0031] 

Herein, if combining the image data DV and the audio 
data DA with the associated information, the associated 
information DM is inserted into a data stream of image or 
a header of the data stream when the image data DV and the 
audio data DA are compressed to generate the material data 
DTm as the data stream. For example, when a moving 
picture compression system in Moving Picture Experts Group 
(MPEG) standardized as ISO (International Organization for 
Standardi zation ) / 1 EC ( International Elect rotechnical 
Commission) 13818-2 is utilized, the associated 
information DM may be inserted into a region for extension 
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or the like provided in a picture layer, thereby allowing 
the associated information DM to be inserted on a picture 
basis. Alternatively, when a synchronization multiplexed 
system in MPEG standardized as ISO/IEC 13818-2 is utilized, 
the associated information DM may be inserted into an 
optional field or the like provided in a header in 
Packetized Elementary Stream (PES). 
[0032] 

Further, when SDI format standardized as SMPTE 
(Society of Motion Picture and Television Engineers) 259M, 
Television-10-Bit 4:2:2 Component and 4fsc Composite 
Digital Signals-Serial Digital Interface, in order to 
transfer non-compressed image and audio data, SDTI format 
standardized as SMPTE 305M, Television-Serial Data 
Transport Interface (SDTI), in order to transfer 
compressed image and audio data, or SDTI-CP format 
standardized as SMPTE 326M, Televi sion-S DTI Content 
Package Format (SDTI-CP) that obtained by further defining 
SDTI format is utilized, it may be inserted into a signal 
having each format as UMID data standardized as SMPTE 330M, 
Television-Unique Material Identifier (UMID) . 
[0033] 

UMID is an identifier for uniquely identifying 
material data such as image data and audio data. UMID 
defines Basic UMID or Extended UMID that is identifier as 
being obtained by adding Signature Metadata to Basic UMID. 
Basic UMID includes a label for identifying digital data, 
information for indicating whether or not overwrite or 
editing is performed on the material data, a number for 
distinguishing the material data from each other, and the 
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like. It is configured that signature Metadata includes 
information on material-data-forming date and time, 
correction information (information on difference in time) 
concerning material-data-forming time, location 
information indicative of latitudes, longitudinal, and 
altitude, information on name of organization, and the 
like. Signature Metadata also includes a region for user 
code, to which the associated information DM may be stored, 
for example. 
[0034] 

The Extended UMID thus obtained is inserted into each 
of the signals having various formats. When SDI format is 
utilized, the Extended UMID is inserted into an ancillary- 
data region, as shown in Fig. 4. When SDTI format is 
utilized, the Extended UMID is inserted into an ancillary 
data region except for a header data portion, as shown in 
Fig. 5. When SDTI-CP format is utilized, the data to be 
inserted is inserted into a payload region on an item 
basis. Specifically, as shown in Fig. 6, system item 
composed of information on image and audio, picture item 
composed of image data, audio item composed of audio data, 
and auxiliary (AUX) item composed of other data are 
inserted in sequence. In the system item, regions for 
metadata sets such as Package Metadata set, Picture 
Metadata set, audio Metadata set, and Auxiliary Metadata 
set are provided to allow the metadata to be inserted 
thereinto. This permits the extended UMID to be inserted 
into these regions. 
[0035] 

By the way, the above imaging apparatus 10 alters 
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read-out timing of imaged charge in the imaging unit 12 to 
generate the material data DTm of a predetermined set 
frame rate FRs , thereby allowing the set frame rate FRs to 
be successively altered. If, however, the set frame rate 
FRs may be altered solely in a stepwise, frame-skipping 
allows the material data DTm of a predetermined set frame 
rate FRs to be generated. This is, generating image data 
DVa having a constant frame rate higher than the set frame 
rate FRs and extracting image data of only the set frame 
rate FRs from the image data DVa allows the material data 
DTm of a predetermined set frame rate FRs to be generated. 
Fig. 7 shows a configuration performing such a generation. 
In Fig. 7, like reference numbers refer to like elements 
shown in Fig. 2 and thus, the detailed description thereof 
is omitted. 
[0036] 

Timing generator 182 in control unit 18 generates 
driving signal CRa corresponding to the highest value of 
the set frame rate FRs that is set through user interface 
unit 16 and supplies it to the imaging unit 12. Based on 
the driving signal CRa, the imaging unit 12 generates 
imaged signal, i.e., imaged signal Spa having a fixed 
frame rate FRq higher than a reference frame rate and then 
supplies it to camera processing circuit 131. When the 
set frame rate FRs may be altered up to n times the 
reference frame rate FRr, it generates the imaged signal 
Spa having a frame rate n times the reference frame rate 
FRr, and supplies it to the camera processing circuit 131. 
[0037] 

The timing generator 182 also generates timing signal 
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CTa synchronized with the driving signal CRa and supplies 
it to the camera processing circuit 131 and audio 
processing circuit 132 in signal processing unit 17 and 
effective frame signal generation circuit 183. 
[0038] 

The camera processing circuit 131 supplies image data 
DVa of fixed frame rate FRq generated based on the imaged 
signal Spa to the effective data selection circuit 171. 
The audio processing circuit 132 supplies audio data DAa 
generated by carrying out a sampling based on the timing 
signal CTa of a fixed frequency to the effective data 
selection circuit 171. 
[0039] 

Imaging control circuit 181 generates a set 
information signal CF indicating a set frame rate FRs 
based on operational signal PSa received from the user 
interface unit 16 and supplies it to the effective frame 
signal generation circuit 183. 
[0040] 

The effective frame signal generation circuit 183 
extracts data on a frame basis from the image data DVa 
based on a ratio of the fixed value of frame rate FRq of 
the image data DVa to the set frame rate FRs indicated in 
the set information signal CF and generates extraction 
control signal CC for generating image data DV of the set 
frame rate FRs. The effective frame signal generation 
circuit 183 also synchronizes the extraction signal CC 
with the timing signal CTa and supplies it to the 
effective data selection circuit 171. When the frame rate 
FRq of the image data DVa is n times the reference frame 
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rate FRr and the set frame rate FRs is n/2 times the 
reference frame rate FRr, the extraction control signal CC 
for controlling data extraction on a frame basis is 
generated every other frame from the image data DVa and is 
supplied to the effective data selection circuit 171 
synchronized with the timing signal CTa. 
[0041] 

The effective frame signal generation circuit 183 
further generates frame rate information DM- FRs indicating 
the set frame rate FRs based on the set information signal 
CF and supplies it to the transmitting unit 15. Since a 
number of frames during a period of frame with the 
reference frame rate FRr may be determined according to 
the extraction control signal CC, sub-frame number BN is 
set on a frame during a period of each frame with the 
reference frame rate FRr and this sub-frame number BN is 
also supplied to the transmitting unit 15 as the frame 
identification information DM-BN. 
[0042] 

The effective data selection circuit 171 extracts the 
image data DVa and the audio data DAa of the frame 
indicated by the extraction control signal CC and supplies 
them to the transmitting unit 15 as the image data DV and 
the audio data DA. It is conceivable that the effective 
frame signal generation circuit 183 may supply the 
effective data selection circuit 171 with the frame rate 
information DM-FRs indicating the set frame rate FRs to 
perform frame-skipping on the audio data DAa according to 
a ratio of the set frame rate FRs to a frame rate when the 
audio data DAa is generated. When the frame rate FRq in 



21 



generating the audio data DAa is n times the reference 
frame rate FRr and the set frame rate FRs is n/2 times the 
reference frame rate FRr, the frame-skipping is performed 
on the audio data DAa every other frame. In this case, 
since an interval of the frame-skipping may be shortened 
as compared by a case where performing the frame-skipping 
on the audio data DAa on a frame basis, audio having an 
excellent sound quality may be get based on the audio data 
DA. 

[0043] 

Thus, the image data DVa having a fixed frame 
frequency makes unnecessary the alteration of operation 
frequencies in the imaging unit 12 and the camera 
processing circuit 131 of the signal processing unit 17, 
thereby allowing configurations of the imaging unit 12 and 
the camera processing circuit 131 to be made simpler. 
Since only the data extraction on a frame basis from the 
image data DVa may generate image data DV of the set frame 
rate FRs, generating the image data DV of a predetermined 
set frame rate FRs from the image data Dva is easily 
allowed. 
[0044] 

When the imaging apparatus is provided with video 
memory or an adder and a divider, it may generate the 
image data DV by adding the image data every the 
predetermined frames. In this case, a variable range of 
frame rate in the imaged signal Sp may be limited and the 
frame rate FRs may be successively altered. Adding the 
imaged signal Sp of n frames and dividing the signal level 
by n allows the signal having a frame rate 1/n times the 
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imaged signal Sp to be obtained even if a frame rate of 
the imaged signal Sp is not divided by n. Further, 
successively altering the read-out frequency of imaged 
charge allows the set frame rate FRs to be successively 
altered . 
[0045] 

Figs. 8 and 9 are diagrams showing relationship 
between the image data DV generated by the imaging 
apparatus 10, 10a and the associated information DM. When 
the set frame rate FRs is set to the one equal to or twice 
the reference frame rate FRr as shown in Fig. 8A, the 
associated information DM that includes the frame rate 
information DM- FRs indicating the set frame rate FRs, as 
shown in Fig. 8C, and the frame identification information 
DM-BN indicating the sub-frame number BN, as shown in Fig. 
8D is combined with image data DV as shown in Fig. 8B (in 
Fig. 8B, frame images based on the image data DV are 
shown). It is to be noted that Fig. 8E illustrates a 
relationship between time and the frame images. The frame 
rate information DM-FRs may also include a magnification 
of the set frame rate FRs to the reference frame rate FRr 
in addition to the set frame rate FRs. The frame rate 
information DM- FRs is indicated by the magnification in 
Figs. 8C and follows. 
[0046] 

When the set frame rate FRs is set to the one equal 
to or half as much as the reference frame rate FRr as 
shown in Fig. 9A, the associated information DM that 
includes the frame rate information DM-FRs indicating the 
set frame rate FRs, as shown in Fig. 9C, and the frame 
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identification information DM-BN indicating the sub-frame 
number BN, as shown in Fig. 9D is combined with image data 
DV as shown in Fig. 9B (in Fig. 9B, frame images based on 
the image data DV are shown). Fig. 9E illustrates a 
relationship between time and the frame images. 
[0047] 

Fig. 10 shows a configuration of signal-recording 
apparatus 20 such as a videotape recorder. 
[0048] 

When the material data DTm that combines main data 
indicating image and/or audio with the associated 
information including frame rate information of the main 
data is input, the material data DTm is supplied to 
encoder 211 of signal-recording processor 21 as recording 
data. The encoder 211 produces error-correcting codes and 
performs data-shuf f ling, multiplexing, channel coding, and 
the like based on operation control signal CTM supplied 
from a recording/reproducing control unit 24, which will 
be described later, using the material data DTm thus 
supplied, to generate record signal SW. The generated 
record signal SW is supplied to a terminal "a" of a 
changeover switch 212. 
[0049] 

To the changeover switch 212, a switch control signal 
CTW is supplied from the recording/reproducing control 
unit 24 and operations of the changeover switch 212 are 
controlled based on the switch control signal CTW so that 
the record signal SW can be supplied to recording 
amplifier 213a or 213b connected to terminal "b" or "c" . 
[0050] 
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The recording amplifier 213a amplifies the supplied 
record signal SW and supplies it. to signal-switching 
device 221a of signal recording/reproducing unit 22 
constituting the signal recording means and the recording 
amplifier 213b amplifies the supplied record signal SW and 
supplies it to signal-switching device 221b. 
[0051] 

When recording a signal, the signal-switching device 
221a supplies magnetic head 222a with the signal SW 
supplied from the recording amplifier 213a based on a 
changeover control signal CWR from the 

recording/reproducing control unit 24. When reproducing a 
signal, the signal obtained by the magnetic head 222a is 
supplied to reproducing amplifier 231a of the signal 
reproduction processor 23. Similarly, when recording a 
signal, the signal-switching device 221b supplies magnetic 
head 222b with the signal supplied from the recording 
amplifiers 213b based on a changeover control signal CWR 
from the recording/reproducing control unit 24. When 
reproducing a signal, the signal obtained by the magnetic 
head 222b is supplied to reproducing amplifier 231b of the 
signal reproduction processor 23. 
[0052] 

Thus, supplying the magnetic heads 222a, 222b built 
in a rotating drum (not shown) with the amplified record 
signal SW allows image, audio, and the associated 
information to be recorded on magnetic tape, which is not 
shown. The signal recorded on the magnetic tape is read 
out by the magnetic heads 222a, 222b and is then supplied 
to the reproducing amplifiers 231a, 231b, respectively. 
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[0053] 

The reproducing amplifier 231a amplifies a signal 
obtained by the magnetic head 222a and supplies it to a 
terminal "a" of a changeover switch 232. The reproducing 
amplifier 231b amplifies a signal obtained by the magnetic 
head 222b and supplies it to a terminal "b" of the 
changeover switch 232. A terminal "c" of the changeover 
switch 232 is connected with a decoder 233. 
[0054] 

To the changeover switch 232, a switch control signal 
CTR is supplied from the recording/reproducing control 
unit 24 and operations of the changeover switch 232 are 
controlled based on the switch control signal CTR on which 
the signals output from the recording amplifiers 213a, 
213b, respectively, are selected, so that it can be 
supplied to the decoder 233 as reproduced signal SR. The 
decoder 233 decodes the reproduced signal SR and performs 
data separation, de-shuffling, an error-correcting 
processing, and the like based on operation control signal 
CTM supplied from the recording/reproducing control unit 
24, thereby generating the material data DTm that combines 
main data indicating image and/or audio with the 
associated information including frame rate information of 
the main data and then transmitting them. 
[0055] 

The recording/reproducing control unit 24 is 
connected with interface unit 25 and generates the switch 
control signals CTW, CTR and the operation control signals 
CTM, CTM controlling operations of the encoder 211 and the 
decoder 233, respectively, based on the operational signal 
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PSv supplied from the user interface unit 25 or another 
operational signal PSw from an external apparatus such as 
the editing apparatus 30. The recording/reproducing 
control unit 24 also performs a driving control of a 
magnetic tape or a rotation head. The 

recording/reproducing control unit 24 further generates a 
display signal PH and supplies display unit 26 with it to 
display an operational condition of a video tape recorder, 
and various kinds of information and the like. 
[0056] 

Although a case where the material data DTm is 
recorded on the magnetic tape has been shown in Fig. 10, 
it may be recorded on disk-like optical or magnetic 
recording medium or signal-reproduction device using 
semiconductor storage element. 
[0057] 

The signal-recording apparatus 20 is enough if the 
image data DV and audio data DA are combined with the 
associated information DM and they are recorded or 
transmitted. Thus, the material data DTm to be input is 
not limited to the one having the associated information 
inserted into the data stream, header, or the like, so 
that, for example, it may be supplied through different 
signal lines. 
[0058] 

Combining the main data with the associated 
information DM allows editing processing and variable 
speed reproduction to be easily performed using the data 
generated by the imaging apparatus 10, 10a or the data 
recorded on the recording medium by the signal-recording 
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apparatus 20. 
[0059] 

During an editing process of the data combining the 
main data with the associated information DM, a speed 
available for a reproduction may be set based on the set 
frame rate FRs. This speed is set so that a natural 
reproduction image can be realized through a frame- 
skipping of a predetermined frame period or a frame 
repetition. For example, if a multiple of the set frame 
rate FRs to the reference frame rate FRr is more than one, 
the speed available for reproduction can be set by 
searching for the divisors of this multiple other than one 
and calculating the reciprocals of the searched divisors. 
Namely, if the set frame rate FRs is 10 times the 
reference frame rate FRr, the divisors are of 2, 4, 5, and 
10 and thus, the reciprocals thereof are of 1/10, 1/5, 1/4, 
and 1/2, respectively. Selection of one of the 
reciprocals of divisors allows the speed available for 
reproduction to be set to less than the same speed as 
normal reproduction speed. When the speed available for 
reproduction not less than the same speed as the normal 
reproduction speed is set to positive integer multiples of 
the normal reproduction speed, the frame-skipping periods 
in each of the speeds available for reproduction may be 
equal to each other. 
[0060] 

If a multiple of the set frame rate FRs to the 
reference frame rate FRr is less than one, the speed 
available for reproduction is set by searching for the 
reciprocals of this multiple and calculating the divisors 
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of the searched reciprocals or integer multiples of the 
reciprocals. Namely, if the set frame rate FRs is 1/6 
times the reference frame rate FRr, the divisors of the 
reciprocals are of 2, 3, and 6 and the integer multiples 

of the reciprocals are of 6, 12, 18, , respectively. 

Selection of one of them to set the speed available for 
reproduction allows frame-skipping periods or numbers of 
the frame repetitions in each of the speeds available for 
reproduction to be equal to each other. The speed 
available for reproduction of less than the same speed as 
the normal reproduction speed is set to the one/positive 
multiples the normal reproduction speed, thereby allowing 
numbers of the frame repetitions in each of the speeds 
available for reproduction to be equal to each other. 
Since, however, this is concerned with a repetition of 
images, a lower limit of the speed available for 
reproduction may be set to the same speed as the normal 
reproduction speed . 
[0061] 

Figs. 11 illustrate GUI representations each for an 
editing operation using the associated information. For 
example, in each of the GUI representations, a material 
administration browser 401 as material administration 
display is provided at an upper-left side thereof and a 
story board 402 as reproduction order display is provided 
at an lower-left side thereof. A monitor viewer 403 as 
reproduced image display for displaying an image before or 
after the editing process is provided at a middle of the 
screen and a time line 404 as reproduction time order 
display is provided at a lower-middle thereof. An 



29 



operation control bar 405 is provided between the monitor 

viewer 403 and the time line 404. 

[0062] 

The material administration browser 401 shows a list 
of items of the material data available for editing and 
for each item of the material data, a stamp view (a 
thumbnail view) for indicating a title, a length, contents 
of items of the stored material data is shown. 
[0063] 

The story board 402 serves as a working area for a 
production of contents and arranging the items of material 
data on the reproduced order allows the contents to be 
produced. The monitor viewer 403 represents not only an 
image based on the material data but also a location of 
reproduction and a variable speed bar for indicating a 
variable range of reproduction speed. 
[0064] 

The time line 404 serves as a working area for a 
production of contents according to a more detailed method 
to allocate items of the material data along a time axis. 
The operation control bar 405 represents an operation key 
for reproducing the items of material data and the 
contents arranged on the time line 404. 
[0065] 

When reproducing the selected material data, the set 
frame rate FRs is determined on the basis of the 
associated information DM, and then, such a calculation 
that the reproduction speed FP multiplied by the set frame 
rate FRs eguals a determined value FD is performed and 
reproduction process conditions are determined on the 
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determined value FD. For example, when the reproduction 
speed is the same speed as the normal reproduction speed 
as shown in Fig. 11A and the set frame rate FRs is 10 
times the reference frame rate FRr, the reproduction speed 
FP multiplied by the set frame rate FRs equals the 

determined value FD as (1x10=10). It is to be noted that 
Fig. 12A illustrates images based on the image data DV 
where the set frame rate FRs is 10 times the reference 
frame rate FRr. Fig. 12B illustrates the frame rate 
information DM-FRs for indicating the set frame rate FRs 
of each of the frame images; Fig. 12C illustrates the 
frame identification information DM-BN for indicating the 
sub-frame numbers BN; and Fig. 12D illustrates the 
absolute frame numbers AN of the frame images. Since the 
set frame rate FRs is 10 times the reference frame rate 
FRr, there are 10 frame images of the set frame rate FRs 
within one frame period of the reference frame rate FRr, 
thereby repeating the sub-frame numbers of zero through 
nine . 
[0066] 

When the determined value FD is set to 10, namely, 
FD=10, as shown in Figs. 12E through 12G, the image signal 
Svm is generated using every 10 frames, namely, with the 
image data of nine frames being skipped, thereby allowing 
reproduced image having same speed as the normal 
reproduction speed to be represented on the monitor viewer 
403 based on the image signal Svm. It is to be noted that 
Fig. 12E illustrates the absolute frame numbers AN of the 
images to be displayed; Fig. 12F illustrates the sub-frame 
numbers BN of the images to be displayed; and Fig. 12G 
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illustrates frame images represented by the image signal 

Svm. 

[0067] 

Next, if an operation is carried out such that an 
indicated width of a frame image representation region 
404a in the time line 404 is enlarged along a direction 
indicated by an arrow A or a position of a sliding cursor 
404d represented by a short bold line in a console 
indication 403c for variable speeds in the monitor viewer 
403 is slid on a direction indicated by an arrow B, it is 
treated so that a processing as the slow motion of 
reproduction is performed. For example, herein, when it 
is set to 1/10 times, a reproduction speed indication 403b 
in the monitor viewer 403 is altered to "x 0,1" as shown 
in Fig. 11B. At the same time, a cursor position in the 
console indication 403c for variable speeds is set to a 
tenth times the normal reproduction speed. Since the long 
reproduction time is required, the indicated width of the 
frame image representation region 404a is also enlarged. 
[0068] 

If the speed available for reproduction is set to 
1/10 times the normal reproduction speed as shown in Fig. 
11B, the determined value FD is set to one, namely, 

FD=10x (1/10) =1. When the determined value FD is set to 
one, namely, FD=1, the image signal Svm is generated using 
every FD=1 frame of the image data DV, thereby allowing 
the reproduced images having a tenth times the normal 
reproduction speed to be represented on the monitor viewer 
403 based on the image signal Svm, as shown in Fig. 12A. 
[0069] 
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if an operation is carried out such that an indicated 
width of the frame image representation region 404a in the 
time line 404 is shortened along a direction indicated by 
an arrow B or a cursor position of the console indication 
403c for variable speeds is slid on a direction indicated 
by an arrow A, it is treated that processing as the quick 
motion of reproduction is performed. For example, when it 
is set to twice as much as the normal reproduction speed, 
a reproduction speed indication 403b in the monitor viewer 
403 is altered to "x 2.0" as shown in Fig. 11C. At the 
same time, a cursor position of the console indication 
403c for variable speeds is set to twice of the normal 
reproduction speed. Since the short reproduction time is 
required, the indicated width of the frame image 
representation region 404a is shortened, as shown in Fig. 
11C. 
[0070] 

If the speed available for reproduction is set to 
twice of the normal reproduction speed as shown in Fig . 
11C, the determined value FD is set to 20, namely, 
FD=10x2=20. When the determined value FD is set to 20, 
namely, FD=20, the image signal Svm is generated using 
every 20 frames, namely, with the image data of nineteen 
frames being skipped, thereby allowing the reproduced 
images having twice speed as much as the normal 
reproduction speed to be represented on the monitor viewer 
4 03 based on the image signal Svm, as shown in Figs. 12H 
through 12K. It is to be noted that Fig. 12H illustrates 
the absolute frame numbers AN of the images to be 
displayed; Fig. 12J illustrates sub-frame numbers BN of 
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the images to be displayed; and Fig. 12K illustrates the 
frame images represented by the image signal Svm. 
[0071] 

Thus, by combining the material data of the set frame 
rate FRs as various kinds of the reproduction speeds with 
each other using the associated information DM, it is 
possible to generate predetermined contents-data for 
broadcasting, a distribution, and the like easily. 
Further, when the edited image data is combined with the 
corresponding associated information to generate the 
contents-data, the user side performs the same processing 
as the one performed in the editing apparatus using the 
associated information included in the contents-data, 
thereby allowing the reproduction speed of contents to be 
altered at the user side. For example, if the set frame 
rate FRs is set to the one higher than the reference frame 
rate FRr to generate the material data such as sports 
event casting and then, using the material data, the 
contents-data including the associated information is 
generated, the user can see only a predetermined scene in 
a slow motion with other scenes being seen in normal 
reproduction speed . 
[0072] 

[Effects of the Invention] 

According to the invention, a signal is read out of an 
image pickup device; an image signal based on captured 
image is generated; when generating image data based on 
the image signal, a frame rate of the image data alter to 
a predetermined set frame rate and frame rate information 
for indicating the set frame rate is generated; and the 
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associated information including the frame rate 
information is combined with the image data to transmit 
them. By, thus, using this frame rate information, it is 
possible to perform the variable speed reproduction. 
[0073] 

Since the frame rate of the image data alters by altering the 
timing of reading a signal out of an image pickup device, the 
frame rate of the image data can be freely changed. Since the 
frame rate of the image data alters by performing frame- skipping, 
the configuration of the imaging apparatus can be made simple. 
Further, since the frame rate of the image data alters by 
altering the timing of reading the signal out of the image 
pickup device and performing addition on a frame basis, 
the frame rate of the image data can be freely changed and a range 
of altering the timing of reading the signal out of the 
image pickup device can be limited. 
[0074] 

Since a sub-frame number is added to each of the 
frames of the set frame rates included within a frame 
period of reference frame rate so as to include the sub- 
frame number in the associated information, it is possible 
to perform the frame-skipping on the image data easily 
when performing the frame-skipping on the image data on a 
frame basis . 
[0075] 

Further analog audio signal is sampled to generate 
audio data, a sampling frequency of the analog audio 
signal is controlled based on the set frame rate, and the 
associated information is combined with the image data and 
the audio data to transmit them. Thus, it is possible to 
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perform the variable speed reproduction on the audio in 

addition to the image comforting the image. 

[0076] 

Since a record signal is generated from the record 
data which combines the main data indicating audio and/or 
video with the associated information containing frame 
rate information of the main data and the record signal is 
recorded on the record medium, it is possible to perform 
the variable speed reproduction easily using the frame 
rate information by reproducing the record medium. 
[BRIEF DESCRIPTION OF THE DRAWINGS] 

[Fig. 1] is a diagram for illustrating a 
configuration of contents-providing system. 

[Fig. 2] is a diagram showing a configuration of 
imaging apparatus . 

[Fig. 3] is a flowchart showing an offer operation of 
sub-frame number. 

[Fig. 4] is an illustration illustrating SDI format. 

[Fig. 5] is an illustration illustrating SDTI format. 

[Fig. 6] is an illustration illustrating SDTI-CP 
format . 

[Fig. 7] is a diagram showing a configuration of 
another imaging apparatus. 

[Figs. 8] are diagrams showing relationship (part 1) 
between the image data and the associated information. 

[Figs. 9] are diagrams showing relationship (part 2) 
between the image data and the associated information. 

[Fig. 10] is a diagram showing a configuration of 
signal -re cording apparatus . 

[Figs. 11] are illustrations each showing GUI 
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representation at an editing operation . 

[Figs. 12] are illustrations showing a reproduction 
operation of the image . 

[Description of Reference Numerals] 

10 , 10a...imaging apparatus; 12. ..imaging unit; 13 , 17...signal 
processing unit; 14 , 18...cont rol unit; 15... t ransmitt ing unit; 
16... user interface unit; 19. ..audio input apparatus; 
131...camera processing circuit; 132... audio processing 
circuit; 141 , 181. ..imaging control circuit; 142 , 182...t iming 
generator; 171. ..ef f ective data selection circuit; 
183. ..ef f ective frame signal generation circuit; 
401. ..material administration browser ; 402... story board; 
403...moni t or viewer; 403b. ..reproduction speed indication; 
403c... con sole indication for variable speeds; 40 4... time 
line; 404a...frame image representation region; and 
405. ..operation control bar. 
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[Name o£ the Document:] Abstract 
[Abstract] 

[Problems] To judge how degree of frame rate in the 
imaging easily, thereby enabling the reproduction of 
contents to be easily controlled. 
[Solving Means] 

An imaging unit 12 reads a signal out of an image 
pickup device to generate an image signal Sp based on 
captured image. Signal processing unit 13 generates image 
data DV from the image signal Sp. By sampling analog 
audio signal Sin, audio data DA is generated. Control 
unit 14 controls operations of the imaging unit 12 and 
signal processing unit 13 to alter a frame rate of the 
image data DV to a predetermined set frame rate and to 
generate associated information for indicating the set 
frame rate or the like. Transmitting unit 15 combines the 
associated information DM with the image data DV and the 
audio data DA to transmit them as material data DTm. By 
generating the record signal based on the data combining 
the associated information and recording it on recording 
medium, it is possible to perform variable speed 
reproduction on the recorded image and audio easily using 
the associated information. 
[Selected Drawing] FIG. 2 
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